We thank all the reviewers for helpful feedback. We will do our best to answer the reviewers' questions and concerns. Because we could not address all the issues due to the lack of space, we will try to include them in the final version. Although Ravi et al. [28] include these adaptive properties, We will include clearer discussion with prior works in the updated version of the paper. We will publicly release the code and trained models if our paper gets accepted.

Bilevel optimization has arisen as a powerful tool for many machine learning problems such as meta-learning, hyper-parameter optimization, reinforcement learning, etc. In this paper, we investigate the nonconvex-strongly-convex bilevel optimization problem, and propose two novel algorithms named deterBiO and stocBiO respectively for the deterministic and stochastic settings. At the core design of deterBiO is the construction of a low-cost and easy-to-implement hyper-gradient estimator via a simple back-propagation. In addition, stocBiO updates with the mini-batch data sampling rather than the existing single-sample schemes, where a sample-efficient Hessian inverse estimator is proposed. We provide the finite-time convergence guarantee for both algorithms, and show that they outperform the best known computational complexities orderwisely with respect to the condition number $\kappa$ and/or the target accuracy $\epsilon$. We further demonstrate the superior efficiency of the proposed algorithms by the experiments on meta-learning and hyper-parameter optimization.